TV advertising commercials exist in virtually all video data streams, subsidizing some or all of the cost of providing the content to the viewer. The ability to identify where the commercials exist in the video data stream has become an important goal for two main reasons. First, advertisers who pay to place the commercials wish to verify that the commercials were actually played, either by being “aired” during a broadcast, or “streamed” during an internet-based viewing session. The auditing process can be greatly enhanced if commercials can be identified as they are being played so that there can be a recordation to document the airing or streaming. Second, technology built into a video playing device, or executing concurrently with a video playing device, can “skip” commercials, assuming that the location of the commercials can be accurately identified so that no programming is skipped. Some conventional technology for identifying where commercials exist in a video data stream is described in an article entitled “Automatic Detection of TV Commercials” (Satterwhite, B.; Marques, O.; Potentials, IEEE, Volume 23, Issue 2, April-May 2004 pp. 9-12). Satterwhite et al. describes two main categories of methods for detecting commercials, namely, “feature-based detection” and “recognition-based detection.” Feature-based detection uses general characteristics of commercials to detect their possible presence. Recognition-based detection works by trying to match commercials with ones that were already learned. Some general characteristics (heuristics) of commercials and commercial breaks include the following:
i. Multiple frames of black are displayed at the beginning and end of each commercial block and between each commercial in the block. There is no audio during these frames.
ii. If a network displays a logo in the corner of the screen, the logo will not appear during the commercials.
iii. Duration is typically some increment of 15 seconds, up to 90 seconds.
iv. Commercials are high in “action,” measured by a relatively larger number of cuts per minute between frames compared to a TV show.
v. Commercial breaks tend to occur at the same time or near the same time in each episode of a given TV series.
Recently, a third reason has arisen to identify where the commercials exist in a video data stream. Mobile devices (e.g., tablets, smartphones) are now in heavy use while viewers watch television (TV). This provides a new platform for synchronized advertising delivery, wherein the TV advertiser may extend their reach to the mobile device. For example, when a particular commercial is airing on, or streaming to, the TV, another commercial may be delivered to the mobile device in either near real-time or in a coordinated delayed time. The mobile ad may be for the same or different product or service as shown in the commercial that was aired on, or streamed to, the TV. To implement such a system, the commercial that was aired on, or streamed to, the TV must be able to be instantly identified.
To facilitate such a system, a database of commercials is maintained so that near real-time matching and identification occurs as a commercial is aired or streamed to a TV. However, each day a small, but significant percentage of commercials are new, and thus do not exist in the database. This results in missed opportunities to maximize the potential of synchronized advertising delivery because the content of the commercial being aired or streamed to the TV largely determines what type of ad should be delivered to the mobile device. One method of expanding the database with the new commercials is to obtain them from advertisers, ad agencies, broadcast networks, and the like. However, this is a laborious process and many of these entities will not provide the necessary information. Accordingly, there is a need to automatically detect unknown commercials directly from a video data stream the first time that they are aired or streamed, so that the database can be very quickly updated.